In case of data corruption or system failure at a primary database, a copy of the primary database may be maintained as a separate database known as a standby database. Thus, if the primary database fails, a failover to the standby database may be performed. Typically, the primary database and the standby database are maintained in separate database systems that are remotely connected. Maintaining consistency between the primary database and the standby database involves replicating changes to the primary database on the standby database.
Furthermore, a database may reside in main memory and/or on disk. A growing trend is to execute queries against main memory databases known as in-memory databases. Typically, all or part of a disk-based database is stored in main memory for relatively faster access to data. Additionally or alternatively, data may be stored in main memory in a different and independent format from data stored on disk. For example, data may be stored on disk in a row-based format, whereas data may be stored in main memory in a column-based format.
Hereinafter, a format in which data is stored on disk is called a persistent format (PF), and a different format in which data is stored in main memory is called a mirror format (MF). Thus, PF data is stored in persistent storage and/or a cache of persistent storage data. In contrast, MF data is stored in main memory separately from any cache of PF data. Advantageously, certain operations, such as vector processing, may be more efficiently performed over MF data as opposed to over PF data.
Because the standby database maintains a copy of the primary database, the standby database is an excellent candidate for sharing some of the primary database's workload. For example, read-only queries may be executed against the standby database instead of against the primary database so that the primary database is available for queries that update the database data. Also, a standby database may maintain MF data in a manner similar to the primary system. The MF data maintained by a standby database is generally not kept in synch with the MF data being maintained by the primary database. In other words, the standby database converts PF data to MF data in any way that enables faster query execution on the standby system.
Maintenance of MF data on a standby database requires maintaining the MF data transactionally consistent with the PF data based on the change records being received from the primary database. Furthermore, a reference timestamp that indicates a time at which the standby database is current with the primary database is generally advanced in discrete steps. As such, for purposes of maintaining the MF data transactionally consistent with the PF data on the standby database, it becomes necessary for the standby system to buffer records from transactions being committed on the primary system until the reference timestamp for the standby database advances to a higher value than the commit timestamp of the buffered transactions.
Such operations on a standby system involve diverse patterns of demand for processing power and storage, which poses significant challenges in memory management for standby systems to ensure that maintaining the MF data is performed efficiently. Because buffering transaction data involves transactions of a wide range of sizes, storage management and preventing fragmentation is a particular problem given the wide range of memory size needs. Also, such standby databases face scalability issues in that hundreds of processes, potentially across multiple database server instances implementing the standby system, could be mining change records from the primary database and also buffering the records from transactions simultaneously.
Furthermore, many transactions perform operations on the same set of blocks multiple times, resulting in the potential for redundancy among the buffered records and inefficiency in applying those buffered records to the MF data. Also, immediate garbage-collecting after freeing of memory chunks can lead to thrashing of memory allocations, especially when the workload peak utilization is unstable and frequently fluctuates, as is common in standby databases.
As such, it would be beneficial to manage memory and resources, for a standby system that supports MF data, to allow for scalable and efficient use of storage resources without significantly retarding the application of change records to the PF data of the standby database.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.